Having thus described the preferred embodiments, the invention is now 

claimed to be: 

1 . An image segmentation process, comprising the following step: 

a) segmenting an image into a main background, a local background, 
and a first object, wherein the first object is on either the main background or the local 
background. 

2. The process set forth in claim 1, step a) further comprising the 
following steps: 

b) identifying a main background for the image; 

c) segmenting a first object on the main background; 

d) classifying the first object into one class of a group of classes 
consisting of a text object, a picture object, and a composite object; 

e) if the first object is classified as a composite object, identifying a local 
background for the first object; 

f) segmenting a next object on the local background; 

g) classifying the next object into one class of a group of classes 
consisting of a text object, a picture object, and a composite object; and 

h) if the next object is classified as a composite object, performing steps 
e) through g) for the next object. 

3. The process set forth in claim 2, step b) further comprising the 
following steps: 

i) identifying one or more block of pixels in the image, wherein the 
dynamic ranges of each of the three color channels are less than a predetermined 
threshold for the block; 

j) applying a region growing algorithm to each uniform block, wherein 
adjacent pixels with a color difference from the uniform block less than a predetermined 
threshold are marked as belonging to the region; 

k) calculating a mean color for each grown region; 

1) combining regions of similar mean color; and 
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m) identifying the region with the largest bounding area as the main 
background for the image. 

4. The process set forth in claim 2, step c) further comprising the 
following steps: 

i) detecting edges in the main background in both vertical and horizontal 

directions; 

j) removing the edge pixels from the main background; 
k) identifying a non-text object on the main background using a connect 
component analysis; and 

1) segmenting the non-text object on the main background. 

5. The process set forth in claim 2, wherein the first object is 
classified as a composite object, step e) further comprising the following steps: 

i) detecting edges in the first object; 

j) removing the edge pixels from the first object; 

k) applying a region growing algorithm to each pixel on the outer 
boundary of the removed edge pixels, wherein adjacent pixels with a color difference 
from the region less than a predetermined threshold are marked as belonging to the 
region; and 

1) identifying the region with the largest bounding area; 

m) comparing the dimensions of the largest region to a predetermined 

threshold; and 

n) if the dimensions of the largest region are greater than the 
predetermined threshold, identifying the region as the local background for the first 
object, otherwise, reclassifying the first object as a picture object. 

6. The process set forth in claim 2, steps f) and g) further 
comprising the following steps: 

i) applying a one-dimensional morphological algorithm to the local 
background in the horizontal direction to segment the next object and classifying the 
next object as a text object if the width of the elements detected by the algorithm is less 
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than a maximal width; and 

j) applying a one-dimensional morphological algorithm to the local 
background in the vertical direction to segment the next object and classifying the next 
object as a text object if the width of the elements detected by the algorithm is less than 
a maximal width. 

7. The process set forth in claim 2, step f) further comprising the 
following steps: 

i) detecting edges in the local background in both vertical and horizontal 

directions; 

j) removing the edge pixels from the local background; and 
k) segmenting the object on the local background using a connect 
component analysis. 

8. An image segmentation process, comprising the following steps: 

a) performing low-resolution segmentation of an image into a main 
background, a local background, and a first object, wherein the first object is on either 
the main background or the local background; and 

b) refining the borders shared by any two of the main background, the 
local background, and the first object at the original resolution of the image. 

9. The process set forth in claim 8, step a) further comprising the 
following steps: 

c) low-pass filtering and decimating the image; 

d) identifying a main background for the image; 

e) segmenting a first object on the main background; 

f) classifying the first object into one class of a group of classes 
consisting of a text object, a picture object, and a composite object; 

g) if the first object is classified as a composite object, identifying a local 
background for the first object; 

h) segmenting an next object on the local background; 

i) classifying the next object into one class of a group of classes 
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consisting of a text object, a picture object, and a composite object; and 

j) if the next object is classified as a composite object, performing steps 
g) through i) for the next object. 

10. The process set forth in claim 9, steps e) and f) further 
comprising the following steps: 

k) applying a one-dimensional morphological algorithm to the main 
background in the horizontal direction to segment the first object and classifying the 
first object as a text object if the width of the elements detected by the algorithm is less 
than a maximal width; and 

1) applying a one-dimensional morphological algorithm to the main 
background in the vertical direction to segment the first object and classifying the first 
object as a text object if the width of the elements detected by the algorithm is less than 
a maximal width. 

1 1 . The process set forth in claim 9, step b) further comprising the 
following steps: 

m) interpolating the low-resolution segmentation results on the original 
resolution of the image; 

n) if any local background is segmented and if any object is classified as 
a picture object, refining the borders of each local background and each picture object; 
and 

o) if any object is classified as a text object, refining the borders and 
interior of each text object. 

12. The process set forth in claim 1 1, step n) further comprising the 
following steps: 

P) applying a region growing algorithm to each of the pixels along the 
borders of each local background and each picture object, wherein adjacent pixels with 
a color difference from the region less than a predetermined threshold are marked as 
belonging to the region 
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13. The process set forth in claim 1 1, step o) farther comprising the 
following steps: 

p) applying a region growing algorithm to each pixel along the border 
of each text object, wherein adjacent pixels with a color difference from the region less 
than a predetermined threshold are marked as belonging to the region; and 

q) applying a region growing algorithm to each pixel of each text object, 
wherein adjacent pixels with a color difference from the region less than a 
predetermined threshold are marked as belonging to the region. 

14. The process set forth in claim 11, further comprising the 
following steps: 

p) determining if the local background is a sweep; and 
q) if the local background is a sweep, reclassifying the composite object 
as a picture object. 

15. The process set forth in claim 14, step p) further comprising the 
following steps: 

r) dividing the local background into 16 sub-regions, wherein the sub- 
regions are in a 4 x 4 arrangement; 

s) computing the mean color of each sub-region; 

t) determining the maximal difference between the mean colors; 

u) comparing the maximal difference to a predetermined threshold; and 

v) if the maximal difference is larger than the predetermined threshold, 
identifying the local background as a sweep. 

16. A method for segmenting an image, comprising the following 

steps: 

a) identifying a main background for the image; 

b) identifying a first object on the main background; 

c) classifying the first object into one class of a group of classes 
consisting of a text object, a picture object, and a composite object; and 

d) if the first object is classified as a composite object, identifying a local 
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background for the first object. 



17. The method set forth in claim 16, further comprising the 
following steps: 

e) if the first object is classified as a composite object, identifying a 
second object on the local background of the first object; 

f) classifying the second object into one class of a group of classes 
consisting of a text object, a picture object, and a composite object; and 

g) if the second object is classified as a composite object, identifying a 
local background for the second object. 

18. The method set forth in claim 17, further comprising the 
following steps: 

h) if the second object is classified as a composite object, identifying a 
third object on the local background of the second object; 

i) classifying the third object into one class of a group of classes 
consisting of a text object, a picture object, and a composite object; and 

j) if the third object is classified as a composite object, identifying a local 
background for the third object. 

19. The method set forth in claim 16, further comprising the 
following steps: 

e) identifying a second object on the main background; 

f) classifying the second object into one class of a group of classes 
consisting of a text object, a picture object, and a composite object; and 

g) if the second object is classified as a composite object, identifying a 
local background for the second object. 

20. The method set forth in claim 19, further comprising the 
following steps: 

h) if the second object is classified as a composite object, identifying a 
third object on the local background of the second object; 
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i) classifying the third object into one class of a group of classes 
consisting of a text object, a picture object, and a composite object; and 

j) if the third object is classified as a composite object, identifying a local 
background for the third object. 

21. The method set forth in claim 16, further comprising the 
following steps: 

e) if the first object is classified as a composite object, determining if the 
dimensions of the local background of the first object are greater than a predetermined 
threshold; and 

f) if the dimensions are not greater than the predetermined threshold, 
reclassifying the first object as a picture object. 

22. The method set forth in claim 16, further comprising the 
following steps: 

e) if the first object is classified as a composite object, determining if the 
local background of the first object is a sweep; and 

f) if the local background of the first object is a sweep, reclassifying the 
first object as a picture object. 
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